A computer program listing may be composed by a programmer as a single linear stream of plain text characters. The text input stream may be converted into tokens in a process referred to lexical analysis. In simple terms, each span of text may be given a single tag that describes the nature of the span, which becomes a fundamental building block in future stages. Through a process referred to as parsing, tokens may be converted into nodes of a parse tree. Tokens that are used by the parser often are represented as terminal nodes in the parse tree. Nodes in the parse tree that are generally not just tokens may be referred to as non-terminals. An abstract syntax tree is a more refined data structure that has enhanced semantic meaning. An abstract syntax tree may assist in enabling semantic analysis, compilation to machine code, transpilation, and other related processes. Formal grammars may be employed to aid in generating code for converting from text, to token, to parse tree, to abstract syntax tree.